Appl. No. 10/764,835 



PATENT APPLICATION 

Attorney Docket No. 139.1006.03 



CERTIFICATE OF TRANSMISSION 
I hereby certify that this correspondence is being transmitted via the Internet to 
the United States Patent and Trademark Office on May 8, 2008, in all-electronic, 
digital form, using the United States Patent and Trademark Office EFS-Web 
system. 

Typed or printed name of person signing this certificate 
Jonathan A. Small 

e-Signature: /Jonathan A. Small/ 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Application of: Julien T. Nguyen 
Appl. No.: 10/764,835 
Filed: 1/26/2004 



Art Unit: 2176 
Confirmation no.: 6888 
Examiner: James T. Debrow 



Title: Multimedia Communication and Presentation 



Filed via EFS-Web 

Assistant Commissioner for Patents 
Washington, D.C. 20231 



REPLY BRIEF 



Sir: 

Applicant respectfully submits this Reply Brief in the appeal from the Office Action dated 
April 12, 2007, (hereinafter referred to as the "Final Office Action") in which all claims were 
finally rejected. Applicant's Notice of Appeal was filed on July 12, 2007, its Appeal Brief was 
filed on December 12, 2007, and the Examiner's Answer was mailed on March 20, 2008. 
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Introduction 

Appellant acknowledges and appreciates the careful consideration given by the 
Examiner to the arguments presented in Appellant's Appeal Brief, filed December 12, 2007. 
The undersigned has reviewed the Examiner's Answer and offers the following reply, 
specifically and limited to certain points raised in that Answer, with each point addressed on its 
own, below. 

1 ) The Examiner's argument that replacing text with an icon is common knowledge is 
insufficient without citation of an actual reference. 

2) The Examiner incorrectly argues that Appellant focuses on the Liles reference, and 
ignores the Maurille and Skelly references with regard to claims 63-70 and 72-78. 

3) The Examiner improperly considers the presence or absence of an icon as an ensemble 
property. 

4) Liles does not teach time taken to review an object being a factor in conditionally 
overriding an ensemble property. 

The arguments presented herein are in addition to and do not supplant any and all 
arguments presented in Appellant's Appeal Brief. Appellant respectfully requests that the Board 
of Appeals consider these points, as discussed in more detail below, when considering the 
Examiner's Answer together with all arguments made by Appellant in its Appeal Brief. 
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1. The Examiner's argument that replacing text with an icon is common knowledge 
is improper without citation of an actual reference. 

In the Examiner's Answer the argument is made that "It has been established and it [is] 
well known in the art that in displaying the emotions, along with associated gestures and 
expressions, and acronyms the system substitutes the appropriate emoticon for the input text." 
(Page 29, lines 17-19.) Furthermore, the Examiner argues that if such were not the case, when 
a system encounters characters that are intended to represent a smiling face, such as :) and 
inserts ©, the system would produce :) ©, which would be redundant. In essence, the Examiner 
is arguing that, at the time of the present invention , it was common knowledge to identify 
portions of a text and replace those portions with a multimedia object within the text. That such 
must be argued to be common knowledge follows from the fact that none of the references cited 
by the Examiner show this, yet it must be shown to make a case of prima facie obviousness of 
the claims of the present application. 

Effectively, the examiner seeks to take "official notice" of the fact that replacement of text 
with an icon, within the text, was well known at the time of applicant's invention. However, as 
noted in In re Ahlert . 424 F.2d 1088, 1091, 165 USPQ 418, 420 (CCPA 1970), the notice of 
facts beyond the record which may be taken by the examiner must be "capable of such instant 
and unquestionable demonstration as to defy dispute". As stated at MPEP 2144.03(A), "It 
would not be appropriate for the examiner to take official notice of facts without citing a prior art 
reference where the facts asserted to be well known are not capable of instant and 
unquestionable demonstration as being well-known." (Emphasis original.) Appellant asserts 
that what was known in terms of in-context replacement of text with multimedia objects (such as 
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those with ensemble properties) at the time of the present invention, and at the latest its date of 
filing (January of 2004), has not been articulated, is uncertain, and should be the subject of 
actual evidentiary support as opposed to mere speculation. Accordingly, official notice is not 

appropriate in this case. 

Furthermore, it is never appropriate to rely solely on "common knowledge" in the art 
without evidentiary support in the record, as fundamental evidence upon which a rejection was 
based. In re Zurko , 258 F.3d 1379, 1385, 59 USPQ2d 1693, 1697 (Fed. Cir. 2001); In reAhlert . 
424 F.2d at 1092, 165 USPQ at 421. One of the fundamental elements of the claims which 
applicant has repeatedly argued is that the cited references do not teach in-context replacement 
of text with one of a group of multimedia objects (in addition to other features such as ensemble 
properties, etc.) Thus, with regard to claims 61, 62, and 71, the subject of the official notice is a 
fundamental element of evidence upon which the rejection was based, and is therefore not 
appropriate. 

Finally, if applicant challenges a factual assertion as not properly officially noticed or not 
properly based upon common knowledge, the Examiner must support the finding with adequate 
evidence. MPEP 2144.03(C). See also 37 CFR 1.111(b). Applicant respectfully asserts that, at 
a minimum: 

detecting whether a mnemonic name is present in said sequences of keystrokes, 
mouse actions, or keystrokes and mouse actions, said mnemonic name being 
associated with one of a set of multimedia objects; 

and 
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when said mnemonic name is detected in said set of sequences, replacing said 
mnemonic name witli said one multimedia object in said set of sequences, 

(claim 61, lines 4-8, and similar language from claim 71) was not a matter of immediate 
common knowledge. 

Accordingly, Appellant respectfully asserts that the Examiner's argument that "the 
combination of Maurille and Skelly teaches each and every limitation found in claims 61, 62, and 
71" (Examiner's Answer, page 21, lines, 17-19) is not factually correct. As neither reference 
shows in-context replacement of text with multimedia objects (asserted to be common 
knowledge), no prima facie case of obviousness has been made as to claims 61, 62, and 71. 
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2. The Examiner incorrectly argues tliat Appellant focuses on the Liles reference, 
and ignores the IVIaurille reference and the Skelly references with regard to claims 
63-70 and 72-78. 

The Examiner's Answer appears to assert that with regard to the rejection of claims 63- 
70 and 72-78, Appellant's Brief focuses on the Liles reference and does not address the 
Maurille or Skelly references (citing Appellant's comments about Liles on pages 16-17, then 
stating that "one cannot show nonobviousness by attacking references individually where the 
rejections are based on combinations of references." Examiner's Answer, page 22, lines 4-10). 
Appellant respectfully disagrees. While it is true that the paragraph bridging pages 16 and 17 of 
the Appeal Brief focuses on Liles, prior to the cited section of text Appellant demonstrated that 
both Maurille and Skelly fail to teach the limitation of replacing a mnemonic name with at least 
one corresponding multimedia object in said set of sequences . See, e.g.. Brief, page 13, lines 
14-15, page 14, lines 7-10. Therefore, when read in context, what Appellant is actually arguing 
is that Liles fails to add to the combination of Maurille and Skelly the limitation of replacing a 
mnemonic name with at least one corresponding multimedia object in said set of sequences. 
As, as each of the three references fail to teach or suggest such a limitaiton . it is logically 
impossible for the combination of those references to teach such a limitation. 

If it would lend added clarity to Appellant's Brief, Appellant would be willing to amend its 
Brief to more explicitly make this argument, for example by reiterating the previous arguments 
as to what Maurille and Skelly fail to teach, that Liles fails to add the missing element, and that 
as all three cited references fail to suggest the limitation, it is logically not possible for the 
combination to teach such a limitation. 
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3. The Examiner improperly considers tlie presence or absence of an icon as an 
ensemble property. 

In claims 67 and 76 there is found the limitation "conditionally overriding at least 
one... ensemble properties in response to a capability of... recipient's presentation device." 
(E.g., claims 67, lines 1-3.) At page 25, lines 12-15, of the Examiner's Answer it is argued that 
Liles teaches that "if a participa[nt] in a chat session has not downloaded the customized bitmap 
file of the user when the user joins the chat session, the participant will see an amorphous 
ghost-like image that represent[s] the user." As more fully discussed below, Appellant 
respectfully points out that there is a significant difference between responding to limitations of a 
user's device and whether a bitmap has been downloaded. Furthermore, Appellant argues 
below that the Examiner mistakenly equates an ensemble property with the presence or 
absence of an image bitmap. 

One feature of the present invention is the ability to present an appropriate multimedia 
object in place of associated text, taking into account the capabilities of the user's device in 
selecting the attributes of the multimedia object to be presented. One example for doing so is 
discussed at page 21 , lines 9-16, of the specification as filed, in which a client receiving a 
multimedia object for replacement of associated text selects for display one of several different 
versions of the multimedia object, such as a stylized graphic of a flower, a relatively small 
photographic picture of a flower, a relatively large photographic picture of a flower, an animation 
of a flower, etc. based on the capabilities of the user device to display same. This is to be 
contrasted with merely representing a user as a ghost-like image if the user's bitmap image has 
not been previously downloaded. Importantly, the later does not suggest the former. 
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Furthermore, the concept of a multimedia object having an ensemble of properties 
means that for each object there are a variety of attributes. These various attributes are 
selectable by the user based on a variety of different needs or interests, such as size of an 
object, memory or display space to be used by an object, animation or static image of an object, 
etc. Appellant respectfully argues that the fact that a system replaces a missing bitmap with a 
ghost image, as attributed to Liles in the Examiner's Answer, does not suggest this concept of 
ensemble properties associated with an object . That is, it does not suggest that each 
multimedia object has its own ensemble properties. In fact, the entire point of presenting the 
ghost-like image is that the object itself is missing from the display device . If the object is 
missing, then the display cannot be a display of the object based on a selection of its properties, 
but is rather a display of a substitute object based on a setting in the display device (i.e., it is not 
a selection of properties of the object per se). And, it appears from Liles that the same ghost- 
like object is displayed regardless of which user bitmap is missing. That is, the nature of the 
ghost-like image (e.g., its outline shape, fill, color, animation, etc.) is not tied in any way to a 
particular user bitmap, but is generic for any and all such objects. Therefore, Appellant 
respectfully argues that the ensemble properties disclosed and claimed in the present 
application are fundamentally different than the step of replacing a missing bitmap with a ghost- 
like image, and therefore the combination of Maurille, Skelly, and Liles fail to render claims 67 
and 76 prima facie obvious. 
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4. Liles does not teach time taken to review an object being a factor in conditionaiiy 
overriding an ensemble property. 

The Examiner has asserted that the Liles reference teaches a "time cue" which is 
analogous to the time feature recited in claims 69 and 78, and accordingly rejects claims 69 and 
78 under the combination of IVIaurille, Sl<elly, and Liles. As discussed further below, Appellant 
respectfully disagrees that the "time cue" from Liles is analogous to the time feature of the 
rejected claims, and that since neither Maurille nor Skelly teach or suggest such a time element, 
the combination of references fails to render claims 69 and 78 prima facie obvious. 

Initially, it should be appreciated that the time element of claims 69 and 78 serves to 
mal<e a decision regarding the selection for display of one of a plurality of ensemble properties 
of a multimedia object. For example, assume that the text "flower" is to be replaced with a 
multimedia object representing a flower, and the object has two different ensemble properties 
representing a static image of a flower and animated image of that flower. According to an 
example of the embodiment claimed in claims 69 and 78, the ensemble properties may be set to 
display the animated image of the flower unless the time taken to review the object is longer 
than a certain threshold, in which case the ensemble property representing the animated image 
is overridden with the ensemble property representing the static image, resulting in the display 
of the static image. 

This is to be contrasted with the "time cue" from Liles. According to Liles, "[t]he Time 
cue is a time interval, which is measured in milliseconds, running from the time that the gesture 
is initiated on a participant's monitor. The Time cue parameter specifies the time that a specific 
avatar frame of the frames in the avatar bitmap file should be displayed." (Col. 8, lines 13-18.) 
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That is, the time cue in Liles represents how long a frame for an avatar animation is displayed 
before the next sequential frame for that Avatar is displayed. 

Appellant asserts that there is a fundamental difference between, on the one hand, time 
as a factor in determining which among several ensemble properties to select for a multimedia 
object and, on the other hand, time during which a frame of an animation is displayed. 
Appellant further asserts that it would not be apparent to one skilled in the art that the time 
during which a frame is displayed could suggest using time taken to review as a mechanism for 
selecting among different attributes of the object to display. Accordingly, Appellant asserts that 
Liles does not teach the claim limitations from claims 69 and 78. As there is no suggestion that 
Maurille nor Skelly teach such a feature, that feature is therefore missing from the combination 
of Maurille, Skelly, and Liles. Finally, it therefore follows that the combination of Maurille, Skelly, 
and Liles must fail to render claims 69 and 78 prima facie obvious. 
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5. Summary and Conclusion 

In summary, the applied reference fails to teach each limitation found in the rejected 
claims 61 to 78. Specifically, the combination of Maurille and Skeily as well as the combination 
of Maurille, Skeily, and Liles falls short of teaching or suggesting the present invention by failing 
to teach, inter alia: 

(a) replacing said mnemonic name with said one multimedia object in said set of 
sequences : 

(b) a theme "being effective to select, for each multimedia object in said ensemble, one 
multimedia object from a set of multimedia object associated with said theme"; 

(c) ensemble properties editable in response to a set of "editing abbreviations"; 

(d) the editing abbreviations comprising a "short sequence of keystrokes or mouse 

actions"; 

(e) "conditionally overriding at least one of said ensemble properties in response to a 
capability of said recipient's presentation device, whereby said multimedia object is 
presented to said recipient in accordance with a different at least one of said 
ensemble properties"; and 

(f) making a decision regarding the replacement of a portion of text with a multimedia 
object based on the "time taken by said recipient to review the multimedia object". 
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Oral hearing has been requested in this appeal. However, if the undersigned can 
personally answer any questions or assist with this case before such hearing, please telephone 
the undersigned at 650-941-4470. 



Respectfully submitted, 



309 2"^ Street, Suite 8 
Los Altos, CA 94022 

Date: May 8, 2008 



/Jonathan A. Small/ 
(e-signature) 

Jonathan A. Small 
Attorney for Applicant 
Registration No. 32,631 
Telephone: 650-941-4470 
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